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center and dispersion, correlation coefficients, statistical significance, 
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by much, on topics that do not comprise the common core. Mean ratings of 
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technical aspects like inferential statistical procedures, were uniformly 
lower for masters' institutions, and somewhat lower for research 
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Introductory Graduate Research Courses: 
An Examination of the Knowledge Base 



Abstract 

This study addresses the question, “What should graduate students know about research 
and statistics after completing an initial course?” Individuals who teach such courses at various 
Carnegie classifications of institutions were surveyed about the specific characteristics of an 
introductory graduate research course at their own institution to see if a core of topics could be 
identified that was common to those courses. Seventeen topics were identified which appear to 
comprise a knowledge base for this course across institutions. These topics may be described as 
those which deal with one’s ability to understand and utilize research results reported in the 
literature. This core includes topics such as the formulation of viable research problems and 
testable hypotheses, the scientific method, types of variables, types of research, null and 
alternative hypotheses, measures of center and dispersion, correlation coefficients, statistical 
significance, reliability and validity to name a few. Institutions differ, although not by much, on 
topics that do not comprise the common core. Mean ratings of importance for non-common core 
topics, most of which deal with technical aspects like inferential statistical procedures, were 
uniformly lower for masters institutions, and somewhat lower for research institutions, than for 
doctoral institutions. Where institutions seem to differ most is in the degree to which topics of a 
more technical nature are addressed within the course and the extent to which such topics may be 
left for inclusion in follow-up courses. 
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Introductory Graduate Research Courses: 
An Examination of the Knowledge Base 



Introduction 

What should graduate students know about research and statistics? The answer to this 
question is quite obviously dependent on a number of variables, including, but not necessarily 
limited to, the degree which the graduate student is seeking, the type of institution in which the 
graduate student is enrolled, possibly the discipline in which the graduate student is majoring, 
and whether or not the student is required to produce some original research as part of those 
degree requirements. Clearly, we cannot provide a simple answer to this question that will apply 
to every graduate student in every situation. 

If we limit our discussion, however, to what we expect a graduate student to know about 
research and statistics after having completed an introductory graduate level research course, 
then regardless of whether or not the graduate student is pursuing a master’s degree or a doctoral 
degree, whether or not the graduate student will be required to write a thesis or dissertation, they 
must start the process with a first course. If we limit our focus to introductory graduate research 
courses, can we now answer the question posed above? That is, what should graduate students 
know about research and statistics after completing an initial graduate research course? 

We attempted to answer this question by examining the introductory graduate research 
course to see if a core of topics could be identified that is common to most or all colleges and 
universities, regardless of whether or not the students are in a masters or doctoral program or 
whether they must write a thesis or dissertation or not. That is, the purpose of this study was to 
determine which topics and/or concepts, if any, in an introductory graduate research course 
comprise a common knowledge base. To accomplish this purpose, we surveyed a number of 
individuals who teach such courses at different types of institutions to obtain from them, the 
specific characteristics of an introductory graduate research course at their institution. 



Background 

At many, if not most, universities, all graduate students are required to complete at least 
one course whose content is devoted primarily to research and research-related topics. While one 
course is generally not sufficient, especially for those students who will be going beyond an M.S. 
Ed. Degree, students generally begin with an introductory course. The students enrolled in such 
a course have a variety of degree and career objectives, including students earning a non-thesis 
master’s degree, master’s students who will write a thesis, doctoral students who must write a 
dissertation but will be practitioners primarily, and doctoral students who will pursue careers as 
researchers. These groups of students have somewhat different needs in terms of what this first 
course offers them. For example, graduate students pursuing master’s degrees may only be 
expected to develop skills that will enable them to critically read and evaluate research in their 
respective disciplines and to identify suitable research problems. If master’s students are 
required to write a thesis, the expectations would be quite different, perhaps including the ability 
to design and conduct a study and to analyze and interpret their findings. Graduate students 
pursuing doctoral degrees would almost always be expected to conduct an entire research study 
from beginning to end. 
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This complete research process involves both technical skills, such as organizing 
literature searches and writing in a scholarly fashion, and critical thinking skills, such as 
proposing thoughtful and appropriate hypotheses based on and extending an existing knowledge 
base. According to Ary, Jacobs, and Razavieh (1996), desired competencies for beginning 
researchers include developing viable research problems; understanding the role of previous 
research and theory; using sampling procedures, statistics, and measurement that are appropriate 
for selected research problems; recognizing the advantages and disadvantages of differing 
approaches to research; interpreting research findings; and writing a scholarly report of those 
findings. Although all graduate students may not be required, in the end, to be proficient in all 
the same areas, many of these students will begin their journey into the world of research in the 
same place, enrolled in the same section of an introductory graduate research course. 

It is also true that some universities tend to train students who primarily come from one 
of these groups more than the others. For example, research and doctoral institutions may be 
more likely to enroll students who will be conducting original research of their own than would 
an institution that offered only master’s degrees. And even in comparing research institutions 
with doctoral institutions, it is likely that the focus would be at least somewhat different. 

Because of the differences between different types of universities, it would be expected that the 
structure of such a course would vary, at least somewhat, from institution to institution. Those 
offering master’s degrees only, may have only a single course devoted to research (Doak, 1982), 
whereas doctoral granting institutions frequently have a sequence of research and statistics 
courses that students are required to take. With the influx of emphasis on alternative 
methodologies and paradigms, some institutions have added additional courses devoted to these 
methodologies and paradigms (Lapan, 1995), while also calling for increased emphasis on the 
value of integrating science with educational practice in graduate research education (Martin, 
1995). Although not necessarily advocating the need for a core curriculum that defines common 
training as the American Psychological Association developed in 1976 (Ellis, 1992), and re- 
asserted in their program accreditation guidelines (APA Committee on Accreditation, 1994), or a 
set of standards as developed by the National Council of the Teachers of Mathematics (1989), it 
may still be expected that a common core of topics and concepts would exist across all 
institutions, with other topics and concepts being more specific to the type of institution. 



Method 

In previous work, an initial list of topics and concepts was developed based upon past 
experiences in teaching introduction to graduate research courses (Young, Moore, Shaw, & 
Mundfrom, 1997). Several instructors of research, statistics, and measurement courses at two 
different universities compiled, examined, and revised the list. A questionnaire (see figure 1) 
was developed using this list of topics, so that the importance of each topic could be assessed, as 
well as the depth to which each topic was addressed. Respondents were also asked to identify 



any topics or concepts not included in the lisl which they deemed to be of importance to an 
introductory research course and the depth to which they would cover these topics in the course 
they taught. They were also asked to identify themselves as being associated with a research, 
doctoral, or master’s institution, and to identify the types of students enrolled in their 
introductory research course. 

The questionnaires were mailed to professors of educational research in numerous 
institutions across the United States and Canada. The colleges selected were limited to those that 
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Figure 1 

For each of the following topics in an introductory graduate research methods course, indicate (1) how important 
you think each topic is to such a course, and (2) the depth to which you cover each topic when you teach such a 
course. Circle one number for Importance and one number for Depth of Coverage for each topic. 



Importance 



Depth of Coverage 



Topic 

Research Process and Design 

Formulation of viable research problems 

Formulation of testable hypotheses 

Scientific method 

Literature sources and searching 

Types of variables 

Types of sampling 

Types of research 

Types of experimental designs 

Ethical and legal issues 

Internal and external validity 

Data Gathering 

Reliability estimates 
Validity estimates 
Standard error of measurement 
Conducting surveys 
Conducting interviews 
Rating scales 
Likert scales 

Descriptive Statistics 

Graphical and tabular displays 
Measures of center 
Measures of dispersion 
Standard scores 
Contingency tables 
Correlation coefficients 
The Normal distribution 

Inferential Statistics 

Elements of probability 
Sampling distributions 
Sampling error 
Significance level 
p-values 
Power 

Type I and Type II errors 
Null and alternative hypotheses 
Interpretation of results 
Statistical vs. practical significance 
Confidence intervals 
t-tests 

Chi-square test of association 
Linear regression 
Analysis of variance 
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offered at least master’s degrees. The professors were asked to evaluate each topic according to 
how important this topic was in the introductory course they taught and the depth to which it was 
covered in their introductory course. 

Questionnaires were mailed to 335 individuals. Of these questionnaires, 33 were returned 
as undeliverable. Of the remaining 302 questionnaires, 80 were returned, with several of those 
accompanied by a comment similar to, “I was unable to complete this form because I no longer 
teach such a course.” While the return rate is low (slightly over 26%), it is likely that a number 
of the non-retumed forms would also fall into the category represented by the above comment. 

We realize that the data gathered do not necessarily constitute a representative sample of 
individuals who teach introductory graduate research courses, however, they do constitute a 
sample of individuals who were concerned enough about the teaching of graduate research 
courses to take a few moments to answer our questions and return the form to us. 

Results 

The principal objective of this study was to determine if a common core of topics exists 
within the introductory graduate research course which could be considered as a common 
knowledge base that transcends different types of institutions. The responses we received appear 
to bear this out. The overall pattern of responses from all individuals is presented in Table 1 . 

Two primary points appear from these data. First, the individual topics which are perceived to be 
of the greatest importance (high percentages in response categories 6 and 7) are ones which are 
essential for an individual to understand if they are going to be an informed consumer of 
research. This group includes most of the topics listed under Research Process and Design, 
along with topics under other headings that are also related to one’s ability to understand and 
utilize the research of others, such as interpretation of results, null and alternative hypotheses, 
statistical vs. practical significance, measures of center and dispersion, correlation coefficients, 
reliability and validity. Second, overall, topics tended to be rated higher under importance than 
they were under depth of coverage. The depth of coverage ratings were generally more diverse, 
with greater percentages being in the lower response categories almost uniformly across the 
board. 

The relative importance of individual topics is somewhat easier to see by examining a 
table of means than through the response patterns. The mean ratings for topics along with their 
standard deviations are presented in Table 2. The topics that are considered to be of the greatest 
importance by the respondents are: the formulation of viable research problems; types of 
research; formulation of testable hypotheses; the scientific method; internal and external validity; 
and types of variables. The next highest rated group of topics in terms of perceived importance 
(based on overall mean rating) consisted of: interpreting results (of inferential tests); literature 
sources and searching; null and alternative hypotheses; statistical vs. practical significance; types 
of experimental designs; measures of center; measures of dispersion; correlation coefficients; 
types of sampling; reliability; and validity. It is these two groups of topics, the highest rated 
topics, that appear to constitute a common knowledge base. As stated above, it would seem that 
a somewhat common thread that ties these topics together, is their close connection with one’s 
ability to understand and utilize the research results reported in the literature. 

The remaining topics were all rated somewhat lesser in terms of perceived importance. 
We divided these topics into two more groups, based on their mean ratings. The third group 
consisted of: ethical and legal issues; significance level; p-values; type I and type II errors; 
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Table 1 

Percentage of all individuals responding in each category 

(n=80) 
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Importance 



Depth of Coverage 



Research Process and Design 


1 


2 


3 


4 


5 


6 


7 


1 


2 


3 


4 


5 


6 


7 


Formulation of viable research problems 




2 


3 


6 


21 


18 


50 




8 


2 


13 


26 


21 


31 


Formulation of testable hypotheses 


2 




5 


8 


19 


24 


42 


2 


8 


10 


3 


21 


27 


29 


Scientific method 






7 


13 


26 


19 


34 


3 


11 


10 


16 


26 


8 


24 


Literature sources and searching 


2 


2 


8 


13 


26 


21 


27 


3 


15 


8 


15 


24 


8 


26 


Types of variables 




2 


3 


16 


18 


34 


26 




5 


10 


19 


31 


13 


21 


Types of sampling 


2 


2 


7 


16 


31 


23 


21 


2 


5 


10 


34 


27 


8 


13 


Types of research 




2 


2 


11 


15 


36 


34 


2 




8 


8 


34 


23 


24 


Types of experimental designs 


2 


3 


8 


18 


23 


32 


15 


2 


8 


18 


18 


26 


19 


8 


Ethical and legal issues 


3 


7 


13 


15 


21 


18 


24 


6 


14 


18 


23 


23 


8 


8 


Internal and external validity 






3 


3 


16 


36 


42 




2 


11 


8 


16 


27 


36 


Data Gathering 






























Reliability estimates 


2 


2 


11 


11 


34 


24 


15 


3 


7 


21 


18 


32 


10 


7 


Validity estimates 


2 


2 


10 


16 


24 


23 


19 


3 


15 


19 


21 


21 


10 


7 


Standard error of measurement 


2 


15 


7 


21 


24 


11 


16 


7 


18 


18 


15 


16 


7 


11 


Conducting surveys 




7 


15 


16 


27 


22 


11 


3 


11 


21 


24 


15 


16 


7 


Conducting interviews 


2 


11 


18 


19 


26 


11 


10 


7 


26 


18 


21 


13 


10 


3 


Rating scales 




2 


16 


29 


31 


13 


8 


2 


13 


36 


18 


15 


11 


3 


Likert scales 




5 


11 


29 


31 


13 


10 


3 


16 


24 


23 


11 


15 


5 


Descriptive Statistics 






























Graphical and tabular displays 
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15 


19 


21 


23 


13 


11 


2 


19 


21 


18 


21 


8 


Measures of center 


7 


3 


7 


13 


15 


32 


24 


11 


2 


11 


19 


16 


24 


15 


Measures of dispersion 


7 


3 


8 


16 


13 


31 


23 


11 


3 


13 


15 


21 


24 


13 


. Standard scores 


10 


8 


8 


19 


23 


18 


11 


15 


7 


10 


24 


23 


16 


7 


Contingency tables 


11 


10 


10 


19 


24 


16 


8 


16 


18 


18 


13 


19 


10 


5 


Correlation coefficients 


7 


3 


5 


10 


15 


34 


23 


11 


2 


10 


24 


15 


23 


13 


The Normal distribution 


13 


8 


5 


10 


24 


16 


24 


14 


10 


14 


18 


15 


16 


13 


Inferential Statistics 






























Elements of probability 


19 


13 


8 


21 


15 


11 


10 


31 


16 


15 


18 


10 


7 


2 


Sampling distributions 


15 


13 


10 


18 


11 


23 


10 


26 


8 


15 


21 


16 


11 


2 


Sampling error 


11 


7 


13 


23 


15 


21 


8 


19 


5 


26 


19 


10 


15 


3 


Significance level 


10 


3 


8 


16 


21 


19 


21 


13 


2 


19 


29 


15 


1 1 


11 


p-values 


10 


7 


11 


16 


13 


18 


26 


13 


8 


19 


26 


8 


13 


13 


Power 


15 


13 


8 


16 


16 


23 


7 


23 


21 


16 


11 


10 


11 


5 


Type I and Type II errors 


11 


3 


7 


16 


23 


16 


24 


11 


8 


16 


21 


16 


15 


13 


Null and alternative hypotheses 


10 


2 


3 


11 


19 


24 


31 


10 


7 


8 


18 


23 


16 


19 


Interpretation of results 


10 


2 


2 


10 


16 


27 


34 


11 




8 


16 


19 


29 


16 


Statistical vs. practical significance 


10 


2 


7 


8 


23 


26 


26 


11 


2 


13 


15 


32 


16 


11 


Confidence intervals 
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13 
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31 
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8 


16 


10 


11 


5 


t-tests 


ii 


15 


7 


18 


16 


24 


10 


19 


15 


13 


16 


13 


16 


7 


Chi-square test of association 


16 


15 


11 


16 


16 


18 


8 


29 


15 


15 


19 


7 


11 


5 


Linear regression 


19 


21 


8 


13 


13 


16 


10 


39 


10 


8 


11 


11 


13 


7 


Analysis of variance 


15 


16 


11 


13 


15 


19 


11 


32 


13 


15 


11 


13 


10 
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Table 2 

Means and Standard Deviations 
For Topics - All Respondents (n= 80) 



Topic 


ImDortance 


Deoth of Coverage 




Mean 


Std. Dev. 


Mean 


Std. Dev. 


Internal and external validity 


6.09 


1.00 


5.63 


1.41 


Formulation of viable research problems 


6.00 


1.24 


5.42 


1.48 


Types of research 


5.85 


1.17 


5.41 


1.32 


Formulation of testable hypotheses 


5.84 


1.33 


5.32 


1.66 


Scientific method 


5.62 


1.27 


4.74 


1.77 


Types of variables 


5.59 


1.23 


5.02 


1.43 


Interpreting results (of inferential tests) 


5.39 


1.84 


4.87 


1.80 


Literature sources and searching 


5.36 


1.45 


4.72 


1.85 


Correlation coefficients 


5.25 


1.73 


4.53 


1.80 


Types of sampling 


5.24 


1.36 


4.59 


1.38 


Null and alternative hypotheses 


5.24 


1.84 


4.63 


1.86 


Measures of center 


5.19 


1.73 


4.60 


1.83 


Validity estimates 


5.15 


1.41 


4.02 


1.54 


Statistical vs. practical significance 


5.13 


1.83 


4.48 


1.73 


Types of experimental designs 


5.11 


1.40 


4.51 


1.49 


Reliability estimates 


5.08 


1.35 


4.28 


1.43 


Measures of dispersion 


5.08 


1.79 


4.55 


1.83 


Ethical and legal issues 


4.93 


1.72 


3.97 


1.62 


Type I and Type II errors 


4.81 


1.91 


4.18 


1.85 


Significance level 


4.80 


1.85 


4.10 


1.75 


Conducting surveys 


4.80 


1.42 


4.13 


1.55 


p- values 


4.72 


1.98 


3.98 


1.86 


The Normal distribution 


4.69 


2.04 


4.08 


1.96 


Likert scales 


4.65 


1.28 


3.88 


1.55 


Graphical and tabular displays 


4.64 


1.66 


4.27 


1.72 


Rating scales 


4.62 


1.20 


3.80 


1.39 


Standard error of measurement 


4.58 


1.67 


3.89 


1.80 


Standard scores 


4.40 


1.79 


4.08 


1.77 


Conducting interviews 


4.33 


1.54 


3.52 


1.57 


t-tests 


4.24 


1.91 


3.64 


1.94 


Sampling error 


4.22 


1.79 


3.53 


1.76 


Contingency tables 


4.18 


1.78 


3.51 


1.79 


O i /4 1 ^ 

UlJU lUUUUiO 


4.06 


i.98 


3.34 


1.79 


Power 


4.03 


1.93 


3.18 


1.88 


Analysis of variance 


4.00 


2.02 


3.08 


1.95 


Chi-square test of association 


3.87 


1.94 


3.13 


1.89 


Confidence intervals 


3.87 


2.12 


3.08 


1.95 


Elements of probability 


3.73 


1.98 


2.85 


1.70 


Linear regression 


3.66 


2.06 


3.11 


2.12 
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conducting surveys; rating scales; Likert scales; the Normal distribution; standard scores; and 
graphical and tabular displays. Group four, the lowest rated topics, consisted of: conducting 
interviews; contingency tables; elements of probability; sampling distributions; sampling error; 
power; confidence intervals; t-tests; analysis of variance; Chi-square test of association; and 
linear regression. These last two groups consist of topics which are either more specialized 
statistical procedures, more closely related to conducting research, or both. It seems that these 
two groups consist of topics which are not part of a common core of topics which exist across 
various types.of institutions, but rather are more specialized topics that would be included in an 
introductory course by some individuals and not by others, depending perhaps, on the types of 
students enrolled in the course. 

Also presented in Table 2 are the mean ratings regarding depth of coverage for each 
topic. As mentioned previously, the depth of coverage responses were uniformly less than the 
perceived importance responses. This same trend is evident in the mean ratings. For the most 
important topics, the ones which were identified as part of a common core, the depth of coverage 
ratings ranged about .4 to .6 points below their corresponding importance rating. For the other 
topics, the depth of coverage ratings were even lower, ranging from .6 to 1.2 points below the 
corresponding importance ratings. 

A secondary question that can be addressed with these data deals with the extent to which 
these ratings varied from one type of institution to another. Respondents were asked to identify 
the Carnegie classification of the institution with which they are affiliated: Research I or II, 
Doctoral I or II, or Masters I or II. The ratings for each of these classifications were compiled 
and examined and it was decided to combine these into three categories: research institutions, 
doctoral institutions, and masters institutions because the differences between the respective I 
and II classifications were either minimal or non-existent. The mean ratings in terms of the 
importance of each topic for the three different types of institutions are presented in Table 3. 
(Note: corresponding tables of response patterns for each type of institution, similar to Table 1, 
are presented in the appendix as Tables 4, 5, and 6.) It is not hard to see that some differences 
exist across these different types of institutions, and that the differences are primarily for the 
topics that do not comprise the common core. 

In general, the mean ratings of importance for the non-common core topics dealing with 
inferential statistics were uniformly lower for masters institutions than for doctoral institutions, 
and also were somewhat lower in the masters institutions than for the research institutions, but 
not by as much. This result is consistent with our initial conjecture that once we identified a 
common core of topics, the different types of institutions would differ on how they perceived the 
importance of other research and statistics topics. 

In terms of comparisons among institutions that enroll education majors versus 
psychology majors or other social/behavioral science majors, the comparison of the mean ratings 
revealed no real differences among these groups or any readily apparent trends in the responses. 
This same lack of differences or apparent trends was also evident when comparisons were made 
among institutions that enrolled students pursuing different types of degrees. The comparisons 
of perceived importance of topics across individuals who taught MS thesis students, MS non- 
thesis students, Ph.D. researchers, and Ph.D. practitioners revealed no obvious differences or 
trends. 

The final question addressed by the respondents was whether any topics were omitted 
that should have been included in this list. There were no responses to this question that were 
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Table 3 

Means for Topics by 
Carnegie Institution Type 



ToDic 


Masters I & II 


Doctoral I & II 


Research I & II 


Internal and external validity 


5.83 


6.18 


6.22 


Formulation of viable research problems 


6.17 


5.81 


6.00 


Types of research 


5.36 


6.12 


5.87 


Formulation of testable hypotheses 


6.17 


6.12 


5.64 


Scientific method 


5.33 


5.62 


5.63 


Types of variables 


5.63 


5.43 


5.67 


Interpreting results (of inferential tests) 


5.00 


5.43 


5.35 


Literature sources and searching 


4.91 


5.50 


5.36 


Correlation coefficients 


5.25 


5.53 


5.12 


Types of sampling 


5.00 


5.43 


5.09 


Null and alternative hypotheses 


5.17 


5.43 


5.12 


Measures of center 


4.75 


5.37 


5.25 


Validity estimates 


5.36 


4.93 


5.16 


Statistical vs. practical significance 


4.67 


5.18 


5.25 


Types of experimental designs 


4.33 


4.93 


5.48 


Reliability estimates 


5.27 


4.75 


5.16 


Measures of dispersion 


4.67 


5.25 


5.12 


Ethical and legal issues 


4.91 


5.56 


4.51 


Type I and Type II errors 


4.50 


5.12 


4.64 


Significance level 


4.42 


5.31 


4.60 


Conducting surveys 


4.82 


4.81 


4.67 


p-values 


4.17 


5.18 


4.64 


The Normal distribution 


4.67 


4.87 


4.54 


Likert scales 


4.75 


4.68 


4.63 


Graphical and tabular displays 


4.33 


4.37 


4.87 


Rating scales 


4.58 


4.75 


4.60 


Standard error of measurement 


4.18 


4.13 


4.86 


Standard scores 


4.25 


4.53 


4.43 


Conducting interviews 


4.00 


4.18 


4.41 


t-tests 


4.00 


4.50 


4.06 


Sampling error 


3.90 


4.40 


4.09 


Contingency tables 


4.09 


3.81 


4.32 


Sampling distributions 


3.83 


4.66 


3.74 


Power 


3.17 


4.35 


4.06 


Analysis of variance 


3.50 


4.37 


3.80 


Chi>square test of association 


3.50 


4.00 


3.70 


Confidence intervals 


3.58 


3.50 


4.06 


Elements of probability 


3.81 


4.40 


3.35 


Linear regression 


3.25 


3.50 


3.77 



ERIC 
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consistently made by all or even a majority of the respondents. The comments that were most 
frequently given dealt with the realm of qualitative research or with specific aspects of 
qualitative methodology. These comments ranged from needing an introduction to qualitative 
research, to qualitative research should be half of the course, to there should be a parallel course 
devoted to qualitative methodology. 

Other comments that were made by more than a single respondent focused on writing, 
such as emphasizing APA style or proposal and report writing; measurement, evaluation, and 
assessment, such as emphasizing dimensions of measurement, formative evaluation, or authentic 
assessment; the use of technology, such as using computers for data collection or for data and 
literature searches. There were a variety of comments, each provided by a single individual, that 
didn’t fit any specific categories, such as provide an overview of research, do a course on meta- 
analysis, emphasize teacher research, and one even calling for more statistical theory. 

There did not seem to be any specific topics or concepts mentioned frequently enough in 
response to this open-ended question that warranted their inclusion in the common core of topics 
identified earlier. The responses were quite varied, indicating, perhaps, that after covering the 
basics, different individuals find different topics that they consider to be important enough to 
devote time to in an introductory course. 

Discussion 

Our objective of identifying a common core of topics in an introductory graduate research 
course seems to have been realized. A list of 17 common topics emerged from our analyses, 
with each of the identified topics having a clear connection to the realm of understanding and 
utilizing published research reports. These appear to be skills that are required of all graduate 
students, regardless of the type of institution or the degree to which the student is aspiring. Even 
within this core of 17 topics, the highest rated topics are those dealing directly with the research 
process such as forming viable problems and hypotheses, the scientific method, types of research 
and variables, and internal and external validity. The remaining topics in this list of 17 are more 
technical in nature and deal with the ability to understand the results as they would be presented 
in a typical research article, such as, measures of center and variation, reliability and validity, 
types of sampling, types of designs, and statistical vs. practical significance. 

Secondarily, we found some differences across different types of institutions, although 
these differences were, in general, not large, and not uniformly present. The most prominent 
differences were among the non-common core topics, particularly those dealing with more 
technical aspects like inferential statistical procedures. These topics were most highly rated in 
doctoral institutions, and least highly rated in masters institutions. One possible explanation for 
these difference patterns may be that in a masters institution, these topics are just not considered 
that important, in research institutions they are often covered in great depth in other courses, 
whereas in doctoral institutions, more importance is placed on these topics as many students may 
not take much in the way of additional coursework in quantitative methodology. 

The comments made in response to the question regarding additional topics had their 
largest focus on the realm of qualitative research. Admittedly, qualitative research and its 
associated methodology were not represented in the initial list of topics that was used to 
construct our questionnaire. Whereas this may appear to be an oversight or a shortcoming, it was 
actually a conscious decision to leave these topics out. That decision was based partly on our 
experiences in teaching this course where qualitative research is introduced but not covered in 
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depth, and our desire to keep the questionnaire as short as it was. Including qualitative research 
as a single topic in our list did not seem to do it justice and including an extensive list of related 
topics would have made the list so long that it may have precluded even the individuals who did 
respond from doing so. We fully expected to get the kinds of responses we got in regard to 
qualitative research, and part of what we were interested in was how broad-reaching those 
responses would be. It would appear that qualitative methodology does not differ much from 
quantitative methodology in regard to its inclusion in an introductory research course— some 
people want a lot, some want a little, some want none at all. 

We realize that these data and our analyses do not represent a definitive work in regard to 
what constitutes a knowledge base of common topics for an introductory graduate research 
course. Our sample was not large, may not have been representative, and our initial list of topics 
may not have included all relevant topics. Consequently, our conclusions should not be viewed 
as THE answer to the question of what topics comprise a common knowledge base for an 
introductory graduate course in research. We do believe, however, that these results can be used 
to aid in the identification of what is typically done in introductory courses, and can also help 
individuals who may be organizing or revising the graduate research curriculum to include new 
or different topics, or when more than one course is offered, to help differentiate between topics 
which should be included in an initial course and topics that can be left to follow-up courses. 
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Table 4 

Master’s Institutions I & II 
Percentage of individuals responding in each category 



Importance 



Depth of Coverage 



minimal 



utmost superficial 



extensive 



Research Process and Design 


1 


2 


3 


4 


5 


6 


7 


1 


2 


3 


4 


5 


6 


7 


Formulation of viable research problems 








8 


17 


25 


50 








25 


17 


25 


33 


Formulation of testable hypotheses 








8 


17 


25 


50 




8 




17 


8 


33 


33 


Scientific method 






8 


17 


33 


17 


25 






8 


50 


25 




17 


Literature sources and searching 




8 


25 


8 


8 


25 


25 


8 


17 


8 


17 




25 


25 


Types of variables 






9 


18 


18 


9 


46 




9 




36 


9 


9 


36 


Types of sampling 






8 


33 


25 


17 


17 






8 


50 


25 


8 


8 


Types of research 








36 


18 


18 


27 








27 


46 


9 


18 


Types of experimental designs 




8 


17 


42 


8 


17 


8 




8 


33 


33 


17 




8 


Ethical and legal issues 


8 


8 


8 


8 


25 


8 


33 


8 


17 




17 


25 


8 


25 


Internal and external validity 








8 


25 


42 


25 






16 


16 


16 


25 


25 


Data Gathering 






























Reliability estimates 






9 


18 


27 


27 


18 






36 


9 


36 




18 


Validity estimates 






9 


9 


36 


27 


18 




18 


36 


9 


9 


9 


18 


Standard error of measurement 


9 


9 


9 


27 


27 


9 


9 




20 


20 


10 


20 


10 


20 


Conducting surveys 




9 


18 


9 


27 


18 


18 






30 


30 


10 


10 


20 


Conducting interviews 


10 


10 


10 


30 


30 




10 




20 


30 


30 


10 




10 


Rating scales 






17 


33 


33 


8 


8 






46 


9 


18 


18 


9 


Likert scales 






17 


33 


25 


8 


17 


9 




36 


9 


9 


18 


18 


Descriptive Statistics 






























Graphical and tabular displays 


8 


8 


17 


17 


17 


25 


8 


25 




8 


17 


25 


25 




Measures of center 


8 


8 


8 


17 


8 


33 


17 


25 






17 


25 


25 


8 


Measures of dispersion 


8 


8 


8 


17 


17 


25 


17 


25 






17 


33 


8 


17 


Standard scores 


8 


8 


8 


25 


25 


25 




25 






25 


33 


17 




Contingency tables 


9 


9 


9 


27 


27 


18 




27 


9 


9 


9 


27 


18 




Correlation coefficients 


8 




8 


8 


8 


50 


17 


25 






8 


17 


33 


17 


The Normal distribution 


17 






25 


17 


25 


17 


17 


8 


8 


8 


25 


8 


25 


Inferential Statistics 






























Elements of probability 


18 


9 




36 


18 


18 




27 


18 


18 


9 


9 


18 




Sampling distributions 


17 


8 


8 


33 


17 


8 


8 


17 


17 


17 


33 


8 


8 




Sampling error 


18 


9 


9 


27 


18 




18 


18 


9 


27 


18 


18 




9 


Significance level 


17 






25 


33 


17 


8 


17 




17 


33 


8 


17 


8 


p-values 


17 


8 


8 


17 


17 


25 


8 


17 


8 


17 


33 




17 


8 


Power 


16 


16 


16 


25 


25 






18 


27 


27 


18 




9 




Type I and Type II errors 


16 






16 


42 


16 


8 


17 




8 


25 


25 


17 


8 


Null and alternative hypotheses 


17 








25 


33 


25 


17 




8 


17 


8 


33 


17 


Interpretation of results 


17 








33 


33 


17 


17 




17 




33 


17 


17 


Statistical vs. practical significance 


17 




8 




33 


33 


8 


17 




17 




42 


17 


8 


Confidence intervals 


25 






50 


17 


8 




33 


17 


25 


8 


8 


8 




t-tests 


25 






17 


42 


17 




18 


9 


9 


9 


36 


18 




Chi-square test of association 


25 


8 


8 


25 


17 


17 




25 


16 


16 


16 


8 


16 




Linear regression 


25 


25 




16 


16 


16 




36 


9 


9 


18 


9 


18 




Analysis of variance 


25 


8 


17 


17 


8 


25 




36 


27 




18 




18 
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Table 5 

Doctoral Institutions I & II 

Percentage of individuals responding in each category 



Topic 


Importance 

minimal 


utmost 


Depth of Coverage 
superficial extensive 


Research Process and Design 


1 


2 


3 


4 


5 


6 


7 


1 


2 


3 


4 


5 


6 


7 


Formulation of viable research problems 




7 




13 


19 


13 


50 




19 




6 


13 


31 


31 


Formulation of testable hypotheses 








12 


12 


25 


50 




6 


13 




6 


38 


38 


Scientific method 






7 


13 


25 


25 


31 


6 


12 


6 


13 


19 


13 


31 


Literature sources and searching 






7 


13 


25 


38 


19 




25 




19 


25 




31 


Types of variables 








19 


25 


50 


6 




6 


13 


6 


38 


25 


12 


Types of sampling 








13 


44 


31 


13 






13 


40 


33 


7 


7 


Types of research 








6 


6 


56 


31 






12 




44 


31 


13 


Types of experimental designs 








19 


19 


12 


50 




12 


19 


19 


13 


37 




Ethical and legal issues 






6 


13 


19 


43 


19 






31 


25 


25 


13 


6 


Internal and external validity 










25 


31 


44 






6 


6 


25 


31 


31 


Data Gathering 






























Reliability estimates 




6 


19 


6 


44 


13 


13 


6 


6 


20 


20 


40 




6 


Validity estimates 


6 


7 


7 


13 


27 


13 


27 


13 


7 


13 


20 


27 


13 


7 


Standard error of measurement 




20 


7 


40 


20 




13 


7 


21 


36 


21 


7 




7 


Conducting surveys 






19 


19 


31 


25 


6 




12 


31 


19 


25 


13 




Conducting interviews 




13 


19 


25 


31 


6 


6 


6 


31 


19 


19 


25 






Rating scales 






13 


31 


31 


19 


6 




12 


38 


25 


19 


6 




Likert scales 






13 


31 


38 


12 


6 




12 


38 


25 


12 


12 




Descriptive Statistics 






























Graphical and tabular displays 


6 




25 


25 


13 


25 


6 


6 


6 


25 


31 


6 


13 


13 


Measures of center 


6 




6 


13 


13 


38 


25 


6 




6 


31 


12 


31 


12 


Measures of dispersion 


6 




6 


13 


19 


38 


19 


6 




12 


19 


19 


37 


6 


Standard scores 


7 


13 


7 


13 


27 


20 


13 


6 


12 


12 


19 


19 


25 


6 


Contingency tables 


19 


13 


13 


6 


31 


12 


6 


19 


31 


13 


6 


19 


6 


6 


Correlation coefficients 


7 


8 






8 


46 


31 


7 


7 




29 


14 


29 


14 


The Normal distribution 


13 


6 


6 




31 


19 


25 


13 


6 


18 


13 


13 


31 


6 


Inferential Statistics 






























Elements of probability 


20 




20 


13 


7 


7 


33 


33 


13 


20 


20 




7 


7 


Sampling distributions 


13 




13 


20 




40 


13 


20 


7 


7 


26 


26 


13 




Sampling error 


7 




20 


27 


20 


20 


6 


13 


7 


33 


27 




20 




Significance level 


6 




6 


19 


19 


12 


38 


6 




18 


25 


18 


18 


13 


p-values 


6 




12 


19 


13 


12 


38 


6 


6 


19 


25 


19 


13 


13 


Power 


14 


7 




29 


14 


29 


7 


27 


27 




20 


6 


6 


13 


Type I and Type II errors 


6 


6 




25 


19 


6 


38 


6 


12 


6 


38 


12 


12 


12 


Null and alternative hypotheses 


6 




6 


12 


19 


19 


38 


6 


6 


13 


6 


31 


13 


25 


Interpretation of results 


6 






19 


19 


25 


31 


6 




12 


19 


31 


19 


12 


Statistical vs. practical significance 


6 






19 


38 


12 


25 


6 


6 




38 


25 


19 


6 


Confidence intervals 


3i 


6 


i2 


ii 


13 


19 


6 


38 


25 




19 


6 


12 




t-tests 


6 


6 


13 


31 


6 


25 


13 


6 


25 


19 


19 


12 


12 


6 


Chi-square test of association 


19 




13 


25 


25 


13 


6 


25 


12 


19 


25 


6 


12 




Linear regression 


31 


6 


6 


25 


13 


6 


13 


44 


12 


6 


13 


6 


13 


6 


Analysis of variance 


12 


6 


12 


19 


13 


25 


13 


19 


12 


31 


6 


19 


12 
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Table 6 

Research Institutions I & II 
Percentage of individuals responding in each category 



15 



Importance 

Tonic minimal 


utmost 


Depth of Coverage 
superficial extensive 


Research Process and Design 


1 


2 


3 


4 


5 


6 


7 


1 


2 


3 


4 


5 


6 


7 


Formulation of viable research problems 






7 


3 


23 


19 


48 




6 




13 


36 


16 


29 


Formulation of testable hypotheses 


3 




6 


6 


23 


26 


36 




9 


6 




36 


19 


26 


Scientific method 






7 


13 


26 


17 


37 


3 


17 


10 


7 


27 


10 


27 


Literature sources and searching 


3 




3 


17 


33 


13 


30 


3 


10 


10 


13 


33 


7 


23 


Types of variables 




-> 




16 


16 


36 


29 




3 


7 


23 


39 


9 


19 


Types of sampling 


3 


3 


9 


13 


29 


19 


23 


3 


9 


7 


29 


29 


9 


13 


Types of research 




-> 


-> 


7 


16 


32 


39 


-> 




7 


7 


26 


23 


36 


Types of experimental designs 


3 


3 




7 


32 


32 


23 


-> 


7 


10 


10 


37 


20 


13 


Ethical and legal issues 


3 


10 


19 


16 


23 


10 


19 


9 


23 


16 


23 


23 


6 




Internal and external validity 






3 


-> 


9 


36 


48 




3 


6 


6 


13 


29 


42 


Data Gathering 
























26 






Reliability estimates 


3 




10 


13 


29 


29 


16 


3 


6 


19 


23 


19 


3 


Validity estimates 






13 


23 


17 


27 


20 




17 


20 


30 


20 


10 


3 


Standard error of measurement 




16 


7 


10 


27 


20 


20 


10 


17 


14 


17 


21 


10 


10 


Conducting surveys 




9 


13 


19 


26 


23 


9 


6 


13 


16 


29 


13 


23 




Conducting interviews 




13 


19 


16 


26 


16 


10 


10 


26 


16 


23 


6 


19 




Rating scales 






13 


30 


33 


13 


7 


3 


17 


33 


20 


13 


13 




Likert scales 




10 


3 


30 


33 


17 


7 


3 


23 


13 


30 


13 


17 
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